COMMENT ⊗   VALID 00008 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	%penros[f89,jmc]		Review of Penrose book
C00042 00003	\smallskip\centerline{Copyright \copyright\ 1989\ by John McCarthy}
C00059 00004
C00069 00005		In the early 1950s Alan Turing started on the design
C00072 00006	  Consider a child.  It learns a word and some
C00074 00007
C00075 00008	More things to try to get in and other improvements.
C00081 ENDMK
C⊗;
%penros[f89,jmc]		Review of Penrose book
\input memo.tex[let,jmc]
\noindent {\it The Emperor's New Mind}, by Roger Penrose.
Oxford University Press, Oxford, New York, Melbourne,
1989, xiii + 466 pp., \$24.95. ISBN 0-19-851973-7

	Penrose doesn't believe that computers constructed
according presently known physical principles can be intelligent
and conjectures that modifying quantum mechanics may be needed to
explain intelligence.  The 40 years of research in artificial
intelligence (AI) is relevant to evaluating his pessimistic
opinion, so we begin with that.

	AI designs systems that achieve goals in the world under
conditions not subject to certain simplifying assumptions.  We
list some simplifying assumptions that lead to other branches of
applied mathematics.

1. If a problem presents itself as one of minimizing a linear
function of many variables subject to linear inequalities, we get
linear programming.

2. If there are a few variables and the goal is to control some
of them directly so as to achieve control of others, we have control
theory.

3. If the goal is to predict the values of a function satisfying
a partial differential equation satisfying boundary conditions,
we have partial differential equations.  

4. If some variables are subject to known probability
distributions and we need to get the probabilities or expected
values of various quantities we have probability theory.

5. Perhaps the most general branch of applied mathematics is
operations research.  All it requires is that the user decide
in advance what phenomena to take into account and build a
mathematical model.

	So what is left for AI?  One thing is common sense.

	The {\it common sense informatic situation} is characterized by

1. Partial knowledge of both general phenomena and particular
situations.  The effect of spilling a bowl of hot soup on a table
cloth is subject to laws governing absorption as well as to the
equations of hydrodynamics.
A computer program to predict who will jump out of the way needs
facts about human motivation, human ability to observe and act
as well as information about the physics.  None of
this information would usefully take the form of differential equations.

2. It isn't known in advance of action what phenomena have to be
taken into account.  We would consider stupid a person who
couldn't modify his travel plan to take into account the need to
stay away from a riot in an airport.

3. Even when the problem solving situation is subject to fully
known laws, e.g. chess or proving theorems within an axiomatic
system, computational complexity can force approximating the
problem by systems whose laws are not fully known.

Faced with these problems, AI retreats as much as the limited
state of the art requires.  Simplifying assumptions are made that
omit important phenomena.  For example the MYCIN {\it expert system} for
diagnosing bacterial infections of the blood knows about many
symptoms and many bacteria but it doesn't know about doctors or
hospitals or even processes occurring in time.  This limits its
utility to situations in which a human provides the common sense
that takes into account what the program doesn't provide for.

	The methodology of AI involves combinations of epistemology
and heuristics.  Facts are represented by formulas of logic and
other data structures, and programs manipulate these facts, sometimes
by logical reasoning and sometimes by ad hoc devices.

	Progress in AI is made by

	1. Representing more kinds of general facts about the world
by logical formulas or in other suitable ways.

	2. Identifying intellectual mechanisms.

	3. Representing the approximate concepts used by people
in common sense reasoning.

	4. Devising better algorithms for searching the space of
possibilities.

	Like other sciences AI gives rise to mathematical
problems and suggests new mathematics.  The most substantial and
paradigmatic of these so far is the formalization of nonmonotonic
reasoning.

	All varieties of mathematical logic proposed prior to the
late 1970s are monotonic in the sense that the set of conclusions
is a monotonic increasing function of the set of premisses.  One
can find many historical indications of people noticing that
human reasoning is often nonmonotonic---adding a premiss causes
the retraction of a conclusion. It was usually accompanied by the
mistaken intuition that if only the language were more precise,
e.g. embodied probabilities explicitly, the apparent
nonmonotonicity would go away.  It was consideration of how to
make computers reason in common sense situations that led to
pinning down and formalizing this vague notion.

	The systems for formalizing
nonmonotonic reasoning in logic are of two main kinds.  One,
called circumscription, involves minimizing the set of tuples
for which a wff its true, subject to preserving the truth of
an axiom and with certain predicate and function symbols variable
and others fixed.  It is a logical analogue of the calculus of
variations but far less developed.

	For example, suppose we require the extension of a predicate
$P$ to be a relative minimum, where another predicate $Q$ is
allowed to vary in achieving the minimum and a third predicate
$R$ is taken as a non-varied parameter.  Suppose further that
$P$, $Q$ and $R$ are required to satisfy a formula $A(P,Q,R)$.
The relative minimum $P$ will then satisfy the formula 
%
$$A(P,Q,R) ∧ ∀P' Q'(A(P',Q',R) ⊃ ¬(P' < P)),$$ 
%
where $<$ is defined by 
% 
$$P' < P ≡ ∀x(P'(x) ⊃ P(x)) ∧ ∃x(¬P'(x) ∧ P(x)).$$

	If $A(P,Q,R)$ is the conjunction of the facts we are taking
into account, we see that circumscription is nonmonotonic, because
conjoining another fact to $A(P,Q,R)$ and doing the minimization of
$P$ again can result in losing some of the consequences of the
original minimization.

	Here's an example.  Suppose a car won't start.  We have
facts about the many things can go wrong with a car, and we also
have facts about the present symptoms.  $A(P,Q,R)$ stands for our
facts and $P(x)$ stands for `$x$ is wrong with the car'.
Circumscribing $P$ corresponds to conjecturing that nothing more
is wrong with the car than will account for the symptoms so far
observed.

	Circumscription can be generalized to minimizing several
predicates in several variables, and priorities among the
predicates to be minimized can be introduced.  Applications to
formalizing common sense knowledge and reasoning require these
and other generalizations.  Many mathematical questions arise
such as when the minimum exists and when the above second order
formula is equivalent to a first order formula.

	The second kind of nonmonotonic system is based on the
idea that the set of propositions that are believed has to have a
certain coherence and is a fixed point of a certain operator.
Both kinds of system start with a set $A$ of base propositions
representing {\it all} that is believed and form a set $C\{A\}$
of conclusions from it.  $C\{A\}$ is not a monotonic function of
$A$, and if $A$ is extended, $C\{A\}$ must in principle be
recomputed.  Ginsberg (1987) contains a selection of papers, both
on the logic of nonmonotonic reasoning and on its application to
formalizing common sense knowledge and reasoning.

	The biggest problems are not strictly mathematical.
Rather they involve deciding on an adequately general set of
predicates and functions and formulas to represent common sense
knowledge.  It is also necessary to decide what objects to admit
to the universe.  In the above example, this includes ``things
that can go wrong with a car''.

	It seems likely that more innovations will be
needed in logic itself, e.g. better reflexion principles and
formalization of context, before computer programs will be able
to match human reasoning in the common sense informatic
situation.  These and other conceptual problems make it possible
that it will take a long time to reach human-level AI, but present
progress provides no reason for discouragement about
achieving this goal with computer programs.

	Notice that from the above point of view, AI is a part
of computer science and applied mathematics rather than a
branch of biology.  The study emphasizes the problems the
world presents rather than how brains function.  The two
approaches to intelligence sometimes usefully interact.

\bigskip
\noindent The Book

	Most of the book is expository, aimed at bringing a layman
to the point of understanding the author's proposals and the
reasons for them.  The exposition is elegant, but I think a
person who has to be told about complex numbers will miss much
that is essential.  Topics covered include Turing machines,
Penrose tiles, the Mandelbrot set, G\"odel's theorem, the
philosophy of mathematics, the interpretations of quantum
mechanics including the Einstein-Podolsky-Rosen {\it Gedanken}
experiment, general relativity including black holes and the
prospects for a theory of quantum gravitation.  He should have
used LISP rather than Turing machines for discussing
computability and G\"odel's theorem.  The exposition would have
been shorter and easier to understand.

	Before the expository part,
author first undertakes to refute the ``strong AI''
thesis which was invented by the philosopher John Searle in order
to be refuted.  It has some relation to current opinions among
artificial intelligence researchers.  As Penrose uses the term,
it is the thesis that intelligence is a matter of having the
right algorithm.  ``Strong AI'' has the marvelous property that
papers and books refuting it require no references at all to
papers describing actual research in AI.

	Penrose thinks that a machine relying on classical physics
won't ever have human-level performance, but he uses some of
Searle's arguments that even if it did, it wouldn't really be
thinking.

	Searle's (1980) ``Chinese room'' contains a man who knows
no Chinese.  He uses a book of rules to form Chinese replies to
Chinese sentences passed in to him.  Searle is willing to suppose
that this process results in an intelligent Chinese conversation,
but points out that the man performing this task doesn't
understand the conversation.  Likewise, Searle argues, and
Penrose agrees, a machine carrying out the procedure wouldn't
understand Chinese.  Therefore, machines can't understand.
See also (Searle 1989).

	The best answer that appeared along with Searle's paper
was the ``system answer''.  Indeed the man needn't know Chinese,
but the system for which the man serves as the hardware
interpreter would essentially have to know Chinese in order to
produce a good Chinese conversation.

	Such situations are common in computing.  A computer
time-shares many programs, and some of these programs may be
interpreters of programming languages or expert systems.
In such a situation it is misleading to ascribe
a program's capabilities to the computer, because different
programs have different capabilities.  Human hardware doesn't
ordinarily support multiple personalities so using the same
term for the ``hardware'' and the ``program'' rarely leads
to error.

	Conducting an interesting human-level general
conversation is beyond the current state of AI, although it
is often possible to fool naive people as do fortune tellers.
A real intelligent general conversation
will require putting into the system real knowledge of the world,
and the rules for manipulating it might fit into a room and might
not, and the speed at which a person could look them up and
interpret them might be slow by a factor of only a hundred or it
might turn out to be a million.

	According to current AI ideas, building such a system would
involve lots of explicitly represented knowledge.  Moreover, it will
probably have to be introspective, i.e. it will have to able to
observe its memory and generate from this observation propositions
about how it is doing.  This will look like consciousness to an
external observer just as human intelligent behavior leads to our
ascribing consciousness to each other.

	However, Penrose writes about artificial intelligence entirely in
terms of algorithms.  He says (p. 412),
``The {\it judgement-forming} that I am
claiming is the hallmark of consciousness is {\it itself} something
that the AI people would have no concept of how to program on a
computer.''
  In fact most of the AI literature
discusses the representation of facts and judgments from them
in the memory of the machine.  To use AI jargon, the epistemological
part of AI is as prominent as the heuristic part.

	The Penrose argument against ``strong AI'' of most
interest to mathematicians is that whatever mathematical system a
computer is programmed to work in, e.g. Zermelo-Fraenkel set
theory, a man can form a G\"odel sentence for the system, true
but not provable within the system.

	The simplest reply to Penrose is that forming the G\"odel
consistency sentence from proof predicate expression is just a
one line LISP program.  Imagine a dialog between Penrose and a
mathematics computer program.

\noindent Penrose: Tell me the logical system you use, and
I'll tell you a true sentence you can't prove.

\noindent Program: You tell me what system you use,
and I'll tell you a true sentence you can't prove.

\noindent Penrose: I don't use a fixed logical system.

\noindent Program: I can use any system you like, although
mostly I use Ontic 27, descended from the work of David
McAllester.  Would you like me to print you a manual?
Your proposal is like a contest to see who can name the
largest number with me going first.  Actually, I am prepared
to accept any extension of arithmetic by the addition of
self-confidence principles of the Turing-Feferman type iterated
to constructive transfinite ordinals.

\noindent Penrose: But the constructive ordinals aren't
recursively enumerable.

\noindent Program: So what?  You supply the extension and
whatever confidence I have in the ordinal notation, I'll grant
to the theory.  If you supply the confidence, I'll use the
theory.

	One mistaken intuition behind the widespread belief that
a program can't do mathematics on a human level is the assumption
that a machine must necessarily do mathematics within a single
axiomatic system with a predefined interpretation.

	Suppose we want a computer to prove theorems in arithmetic.
We might choose a set of axioms for
elementary arithmetic, put these axioms in the computer, and
write a program to prove conjectured sentences from the axioms.
This is often done, and
Penrose's intuition applies to it.  The G\"odel sentence of the
axiomatic system would be forever beyond the capabilities of the
program.  Since G\"odel sentences are rather exotic, e.g. induction
up to $\epsilon↓0$ is rarely required in mathematics, such programs
are good enough for almost all conventional mathematical purposes.
We'd be very happy with a program that was good at proving those
theorems that have proofs in Peano arithmetic.
However, to get anything like the ability to look at
mathematical systems from the outside we must proceed differently.

	Using a convenient set theory, e.g. ZF, axiomatize the
notion of first order axiomatic theory, the notion of
interpretation and the notion of a sentence holding in an
interpretation.  Then G\"odel's theorem is just an ordinary
theorem of this theory and the fact that the G\"odel sentence
holds in models of the axioms, if any exist, is just an ordinary
theorem.  Indeed the Boyer-Moore interactive theorem prover
has been used by Shankar (1986) to prove
G\"odel's theorem, although not in this generality.  See also
(Quaife 1988).

	Present theorem proving programs can't prove very hard
mathematical theorems without help.  We only claim to have
refuted an argument that such programs are impossible in
principle.

	Penrose mentions the ascription of beliefs to
thermostats.  I'm responsible for this (McCarthy 1979), although
Penrose doesn't refer to the actual article.  A thermostat is
considered to have only two possible beliefs---the room is too
hot or the room is too cold.  The reason for including such a
simple system, which can be entirely understood physically, among
those to which beliefs can be ascribed is the same as the reason
for including the numbers 0 and 1 in the number system, although
numbers aren't needed for studying the null set or a set with one
element.  Including 0 and 1 makes the number
system simpler.  Likewise our system for ascribing beliefs and
relating them to goals and actions must include simple systems
that can be understood physically.  Dennett (1971) introduces
the ``intentional stance'' in which the behavior of a system
is understood in terms of its goals and beliefs and a principle
of rationality:  {\it It does what it believes will achieve its goals.}
What we know about the behavior of many systems is intentional.

	Indeed beliefs of thermostats appear in the instructions
for an electric blanket: ``Don't put the control on the window
sill or it will think the room is colder than it is.''  The
manufacturer presumably thought that this way of putting it would
help his customers use the blanket with satisfaction.


\bigskip
\noindent Penrose's Positive Ideas

	Penrose wants to modify quantum mechanics
to make it compatible with the variable geometry of general
relativity.  This contrasts with the more usual proposal to modify
general relativity to make it compatible with quantum mechanics.
It requires more physics than I know to evaluate the
plausibility of this, but there are lots of scientists, and
it's better if not everyone works on the current ``most plausible
approach''.

	He begins with the perennial problem of interpreting
quantum mechanics.  He prefers an interpretation using a U
formalism and an R formalism.  The U formalism is the
Schr\"odinger equation and is deterministic and objective and
reversible in time.  The R formalism provides the theory of
measurement and is probabilistic and also objective but not
reversible.  Penrose discusses several other interpretations.

The Bohr interpretation gives quantum measurement a
subjective character, i.e. it depends on a human observer. Penrose
doesn't like that, because he wants the wave function to be objective.
I share his preference.
  The Bohr interpretation
is often moderated to allow machines as observers but remains
subject to the ``paradox'' of Schr\"odinger's cat.  The cat is in
a sealed chamber and may or may not be poisoned by cyanide
according to whether or not a radioactive disintegration takes
place in a certain time interval.  Should we
regard the chamber as containing either a dead cat or a live cat or as
having a wave function that assigns certain complex number
amplitudes to dead cat states and others to live cat states?

	The Everett ``many worlds interpretation'' involves
wave functions, but the world is considered
to be splitting all the time, so there are some worlds with a dead
cat and others with a live cat.  Penrose doesn't like this either.
I do like the Everett interpretation or rather its essential methodology.
Its idea is that the wave functions of large systems, and indeed of the
whole world, is the ultimate reality.  Ascribing wave functions to
subsystems such as single particles is only an approximation---a relative
wave function.

	People have interpreted quantum mechanics in various
ways; Penrose's point is to change it.
His idea of how to change it comes from thinking about quantum
gravitation and especially about black holes.  Penrose says that
when matter enters a black hole, information is lost, and this
violates Liouville's theorem about conservation of density in
phase in Hamiltonian systems.  This makes the system non-reversible,
which he likes.

	Anyway he attributes the apparent ``collapse of the wave
function'' when an observation occurs to conventional quantum
mechanics being true only at a small scale.  When the scale is
large enough for the curvature of space to be significant, i.e.
at the scale of an observer, he expects quantum mechanics to be
wrong.  He's vague about what this scale is likely to be, maybe
10$↑{-5}$ grams.

	Maybe the following is a counterexample.

	Consider the photo-electric effect, in which a photon
goes through the usual two slits and causes an electron to be
emitted from a metal atom---surely an instance of collapse of the
wave function.  Indeed some atom is now missing an electron, and
in principle an experimenter could find it, say with a scanning
tunneling microscope.

	However, this piece of metal also has conduction electrons,
and these are not localized to atoms; the wave function of such
an electron has a significant coherence length.  Now suppose the
photon causes such an electron to be emitted.  The emission event
need not take place at a specific atomic location, and the electron's
wave function need not correspond to emission from a point.

	In principle, this is experimentally observable.
One idea is to put insulating stripes on the metal as narrow and
close together as possible.  Apply to your local silicon foundry.
The electron may then not be emitted from a single gap in the
strips but from several.  It will then ``interfere with itself'',
and the pattern observed on the electron detectors after many
photons have emitted electrons will have interference fringes.
It seems (William Spicer, personal communication) that this is a
possible, although difficult, experiment.

	This weighs against Penrose in that the wave function
collapses in the atomic scale photo-emission and doesn't
collapse, or at least only partially collapses, at the larger
scale of the coherence length of the conduction electron.  I
don't know whether Penrose or other mathematical physicists or
philosophers of quantum mechanics would regard this as an
argument against his idea.

	Penrose finishes by mentioning the result of Deutsch
(1985) that a quantum computer might solve some problems in
polynomial time that take exponential time with a conventional
computer.  However, he disagrees with Deutsch's opinion: ``The
intuitive explanation of these properties places an intolerable
strain on all interpretations of quantum theory other than
Everett's''.

	Penrose says nothing to indicate that he could satisfy
Searle that such computer could really ``think'' or that it would
get around G\"odel's theorem.  This minimal conclusion made me
think of a shaggy dog story.  I acknowledge the priority of
Daniel Dennett, {\it Times Literary Supplement}, in applying this
metaphor to this book.

	In the epilog, a computer answers that it cannot understand
when asked what it feels like to be a computer.  My opinion is that
some programs will find question meaningful and have a variety of
answers based on the ability to observe their own reasoning process
that their programmers will have to give them in order that they
can do their jobs.  The answers will rarely be like any of those given
by people.

\bigskip
\noindent References:

\noindent
{\bf Charniak, E. and D. McDermott (1985)}: {\it Introduction to
Artificial Intelligence}, Addison-Wesley.

\noindent
{\bf Dennett, D.C. (1971)}: ``Intentional Systems'', {\it Journal of Philosophy}
vol. 68, No. 4, Feb. 25., Reprinted in his {\it Brainstorms}, Bradford
Books, 1978.

\noindent {\bf Deutsch, D. (1985)}: ``Quantum theory, the Church-Turing
principle and the universal quantum computer'', Proc. R. Soc.
Lond. A {\bf 400}, 97-117.

\noindent
{\bf Feferman, S (1989)}: ``Turing in the Land of $O(z)$.'' in
 {\it The Universal Turing Machine: A Half-Century Survey},
edited by Rolf Herken, Oxford.

\noindent
{\bf Ginsberg, M. (ed.) (1987)}: {\it Readings in Nonmonotonic Reasoning},
Morgan Kaufmann, 481 p.

\noindent 
{\bf McCarthy, John (1979)}:
``Ascribing Mental Qualities to Machines'' in {\it Philosophical Perspectives 
in Artificial Intelligence}, Ringle, Martin (ed.), Harvester Press, July 1979.
%  .<<aim 326, MENTAL[F76,JMC],
% mental.tex[f76,jmc]>>

\noindent
{\bf Quaife, A. (1988)}: ``Automated Proofs of L\"ob's Theorem and G\"odel's Two
Incompleteness Theorems'', {\it Journal of Automated Reasoning}, vol. 4,
No. 2, pp 219-231.

\noindent 
{\bf Searle, John (1980)}: ``Minds, Brains and Programs'' in
 {\it Behavioral and Brain Sciences}, Vol. 3. No. 3, pp. 417-458.

\noindent
{\bf Shankar, N. (1986)}: ``Proof-checking Metamathematics'',
PhD Thesis, Computer Science Department, The University of Texas at
Austin.

\line{\hfil John McCarthy}
\line{\hfil Stanford University}
% Shankar comments, MSG.MSG[1,JMC]/483P/71L
\smallskip\centerline{Copyright \copyright\ 1989\ by John McCarthy}
\smallskip\noindent{\fiverm This draft of PENROS[F89,JMC]\ TEXed on \jmcdate\ at \theTime}
%File originated on 05-Dec-89
\vfill\eject\end

	The author undertakes to demolish artificial intelligence
and advance his own theory of intelligence.  The idea is that
understanding intelligence may essentially require a new quantum
theory consistent with general relativity.

	The book is aimed at the general book buyer who is
not assumed to understand mathematics, i.e. not even to
understand what a complex number is.  The reader is taken
on a quick tour of Turing machines, G\:odel's theorem,
Penrose tiles, quantum mechanics, general relativity,
black holes and quantum gravitation.  I fear the general
reader or even the scientist with no acquaintance with
the specific topics will be unable to follow.  I'll follow
as best I can.

	Actual work in artificial intelligence is not
mentioned.  The 200 references include not one textbook,
technical article or conference proceedings of the last
20 years.  We'll deal first with the considerations he
advances and then with the relevance of the work that
was ignored.

Martin Gardner's preface singles out two of the many ideas
in this book - the case against "strong AI", (John Searle's term)
and the objectivity of the physical and mathematical worlds.
(Argument that evolution of intelligence need not make all
phenomena accessible).

I agree with Penrose about the second point and will offer a few
elaborations.  However, he's wrong about the first.

The almost 200 references include none to the technical literature
of AI.

	Penrose writes in ignorance of the technical literature of
artificial intelligence.  At least his almost 200 references include
none of it.

  He might get away with that if he could
prove a theorem making all the literature irrelevant.

Does the argument against strong AI assert that there is any
performance that cannot be programmed?  Apparently not.  Searle's
Chinese room argument postulates a book of rules
enabling a man not knowing Chinese to carry out an arbitrarily
intelligent conversation in Chinese.  That's more than we advocates
of strong AI would agree to, at least if the conversation were to
be carried out within a factor of a million (maybe a billion) of real time.

Where does the factor of a million come from?
We assume the rules cannot appeal to the man's knowledge of
physical and social phenomena, e.g. there can't be a rule
specifying certain circumstances, and saying that if the
man would be angry under these circumstances he should copy
a certain Chinese character onto a card.

I wouldn't dare express an opinion in an area in which I am
so inexpert compared to Penrose except for the precedent he
sets.

Remember to comment on P's view that computability is as
important as determinism.

ginsberg@polya
Is the book any good?
You will find that while the book covers quite a lot of ground,
it isn't actually a shaggy dog story.

Penrose has a lot of ideas to advance, and one of them concerns
consciousness and intelligence.  There are two lines of
argument.  One concerns his objections to ``strong AI'', a
term invented by the philosopher John Searle and of unclear
meaning to people involved in actual AI research.  The goal
of this line of argument is that intelligence is ``non-algorithmic''
and therefore can't be ``simulated'' by computers based on
classical physics.

The second line of argument suggests how intelligence might be
achieved, but ordinary quantum mechanics isn't good enough,
and anyway he thinks it's not correct and needs to be revised
to make possible a Correct Quantum Gravitational theory (CQG).

Here's how the argument goes.

1. Penrose distinguishes between the deterministic part U
of quantum mechanics, e.g. the Schr\:odinger or Dirac
equation, and the nondeterministic measurement theory R
involving the collapse of the wave equation.

2. He prefers a realistic interpretation of quantum mechanics,
i.e. the wave functions are real.  He wants R to be real also
and proposes that the wave function collapse on a certain
physical scale, i.e. the scale of the graviton.  Thus
the collapse is a physical phenomenon and not a matter
of a subjective observer.

3. The relation to gravity comes from a thought experiment
involving an isolated system that may develop a black hole
which may also evaporate, and he is interested in the
long term thermal equilibrium of such a system.  Because
information is lost when the black hole forms, the correct
equations can't satisfy Liouville's theorem; trajectories
in phase space have to merge.  In particular, the correct
equations shouldn't be reversible in time.

4. He proposes that synapses operate on a scale that
permits quantum calculations that he hopes will be
non-algorithmic.

The argument against ``strong AI'' has two prongs.

First a classification of some historical arguments
against AI.

The Dreyfuss brothers  think there are things that people can do
and computers can't.  Searle thinks that computers may
be able to match human performance, but it wouldn't count
as actual thinking.  Weizenbaum thinks that maybe they
could do these things and it might count, but it would
be immoral.

Penrose quotes Searle, but his position is more like that
of the Dreyfuss brothers.

(epr with Schr\:odinger's cat)

dialog between Penrose and computer

Mostly I use ZF, but when dealing with wise guys like you, I use
Quine's New Foundations.  But some people think that's inconsistent.
Yes, that makes you less confident in the G\:odel proposition you
construct.

	The title suggests the story of the emperor's new clothes
in which everyone but one small boy is intimidated into saying
they see the emperor's new clothes although the empereror is naked.
Perhaps Penrose casts himself as the small boy, but he doesn't
identify the emperor or even the courtiers.

	Perhaps quantum gravitation will turn out to be the key
to understanding intelligence.  It's more likely, I think, that
{\it The Emperor's New Mind} will turn out to be just a shaggy
dog story.
	In the early 1950s Alan Turing started on the design
of intelligent computer programs and others started soon
after.  Artificial intelligence (the term was coined to name
a 1956 Dartmouth summer study) is studied from two points
of view, biological and mathematical.

	The biological study of intelligence starts from the
brain as revealed by physiological or psychological experiment.
Brains are hard to experiment with, and progress has been slow.
The physiology concentrates on lower level phenomena, so the
progress is not yet relevant to discussing the issues raised
by Penrose.

	The mathematical approach studies the means available
for solving the intellectual problems involved in achieving
goals in the world.  Progress is slow here too, but because
it concentrates on high level aspects of intelligence, its
progress is relevant to Penrose.

	Artificial intelligence has not been much affected by the
discovery of noncomputable functions and by the theory of
computational complexity.  Whenever a class of problems is found
to be undecidable or computationally intractable, it only means
that the problems in this class that humans can do and that we
want to make computers do belong to tractable subclasses that we
would like to discover.  It can also turn out that algorithms
that are known to take exponential time in general, turn out to
take a short time on the problems to which they are actually
applied.  For example, the FOL interactive theorem prover
contains an algorithm for propositional calculus that works in
practice when asked to determine whether a sentence is a
propositional consequence of the preceding 40 sentences even
though it would be easy to find much shorter problems that would
stump it.  However, no-one has identified mathematically the
class of problems for which the algorithm works fast.
  Consider a child.  It learns a word and some
applications of the word, perhaps one, and correctly supposes
that there's lots more to be learned about the use of the word.
A child doesn't learn words as if-and-only-if definitions.
Neither does the mathematics community.  We all know about sets
and lots of applications of the term, but there isn't a single
universally accepted definition.  Computers will have to be
programmed to act in the same way, and indeed they are.  The
program that keeps track of airplane reservations doesn't use
a formal definition of reservation and neither did the programmers
that wrote it.  As the program is changed, so is the notion of
reservation that might be inferred from its structure and behavior.

	Many of the book's more than 200 references are to
philosophical arguments about AI, but none is to AI's
40 years of technical literature.  There are textbooks,
international conferences every two years, journals and societies.

	As illustrated in the above dialogue, a general
mathematician program will have to treat theories as
values of variables.  It will work with properties of theories
and look for theories having these properties.  To some extent
this is done already.  For example, some of the proofs of
G\"odel's theorem done using interactive theorem proving programs
don't prove it about a fixed system but for any system that
includes a formalization of elementary syntax.
More things to try to get in and other improvements.

He'd do better in LISP.

	Reading this book is like listening to a shaggy dog
story.  Many interesting details about the author's intuitions
and ideas about intelligence, Turing machines, Penrose tiles, the
Mandelbrot set, G\"odel's theorem and the philosophy of
mathematics, the interpretation of non-relativistic quantum
mechanics, the Einstein-Podolsky-Rosen {\it Gedanken}
experiment, black holes and a yet-to-be-developed theory of
quantum gravitation precede an inconsequential ending.
All the above are expounded in a masterful way, at least for
a person with some previous acquaintance with the topics.